[DO NOT MERGE] dimensional os hackathon project - team golden state warriors#2292
[DO NOT MERGE] dimensional os hackathon project - team golden state warriors#2292samlee815 wants to merge 1 commit into
Conversation
samlee815
requested review from
leshy,
mustafab0,
paul-nechifor and
spomichter
as code owners
Greptile SummaryThis PR adds the "greeter-agentic" hackathon demo: a Unitree Go2 that autonomously patrols, detects a user-described target via a VLM, approaches to a safe standoff using lidar/raycast ranging, and performs the Hello wave gesture before resuming patrol. The implementation is well-architected with a clean separation between the pure state machine (
Confidence Score: 3/5Not safe to merge as-is: the PR description contains a live API credential that needs to be revoked and removed, and shared setup docs point to a developer-specific directory that will break for anyone else. The leaked Alibaba API key in the PR description is an active credential exposure that requires immediate remediation regardless of the DO NOT MERGE label. The developer-specific default path in the README and launch script means the documented setup is broken for every other contributor. Beyond those two issues the core robot logic is sound — the state machine, geometry helpers, and threading model are all well-structured and covered by tests. README.md and scripts/run_greeter_sim.sh for the hardcoded personal path; visited_registry.py for the unbounded list growth; skill_container.py for the unguarded base64 decode. Important Files Changed
Sequence DiagramsequenceDiagram
participant MCP as MCP / Operator
participant GSC as GreeterSkillContainer
participant PM as PatrollingModule
participant VLM as QwenChinaVlModel
participant TAM as EdgeTAMProcessor
participant GO2 as GO2Connection
MCP->>GSC: start_greeting(target)
GSC->>PM: start_patrol()
loop wander tick (12 Hz)
GSC->>VLM: detect_all(image, description)
VLM-->>GSC: [bbox, ...]
GSC->>GSC: select_target() via VisitedRegistry
end
GSC->>TAM: init_track(bbox, image)
GSC->>PM: stop_patrol()
loop approach tick
GSC->>TAM: process_image(frame)
TAM-->>GSC: detection (bbox)
GSC->>GSC: _locate() lidar → raycast fallback
GSC->>GSC: cmd_vel.publish(approach_twist)
end
Note over GSC: distance in [min_safe, standoff+tol]
GSC->>GO2: publish Hello gesture
Note over GSC: greet phase (timed, no cmd_vel)
GSC->>VLM: detect_facing(image)
VLM-->>GSC: front / back / None
opt subject faces camera
GSC->>GO2: publish FingerHeart
end
GSC->>GO2: RecoveryStand then BalanceStand
GSC->>GSC: registry.mark_visited(position)
GSC->>PM: start_patrol()
MCP->>GSC: stop_greeting()
GSC->>PM: stop_patrol()
GSC->>GSC: cmd_vel.publish(zero)
Reviews (1): Last reviewed commit: "feat: bulk commit for hackathon implemen..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| From this DimOS checkout: | ||
|
|
||
| ```bash | ||
| export DIMOS_HOME="${DIMOS_HOME:-$HOME/sam/dimos}" |
There was a problem hiding this comment.
The default value $HOME/sam/dimos references a developer's personal directory. Anyone cloning the repo and following this README verbatim will resolve to /home/$USER/sam/dimos, which almost certainly does not exist and will cause a confusing failure. The same path also appears in scripts/run_greeter_sim.sh line 22 and in the "Test" block on line 46. The correct default should use the repo root (e.g., $(pwd) or $REPO_ROOT) or leave it as a required export with no silent fallback.
| def is_visited(self, position: Position, now: float = 0.0) -> bool: | ||
| """Whether ``position`` was greeted within the radius and not forgotten.""" | ||
| return any( | ||
| math.dist(position, seen) <= self._radius | ||
| and not self._expired(when, now) | ||
| for seen, when in self._visited | ||
| ) |
There was a problem hiding this comment.
_visited accumulates every greeting ever recorded and is never compacted. is_visited iterates the full list on every scan, so for a long-running demo the scan cost grows linearly with the number of greetings. Pruning expired entries keeps the list bounded to the active window size.
| def is_visited(self, position: Position, now: float = 0.0) -> bool: | |
| """Whether ``position`` was greeted within the radius and not forgotten.""" | |
| return any( | |
| math.dist(position, seen) <= self._radius | |
| and not self._expired(when, now) | |
| for seen, when in self._visited | |
| ) | |
| def is_visited(self, position: Position, now: float = 0.0) -> bool: | |
| """Whether ``position`` was greeted within the radius and not forgotten.""" | |
| self._prune(now) | |
| return any( | |
| math.dist(position, seen) <= self._radius | |
| for seen, _ in self._visited | |
| ) | |
| def _prune(self, now: float) -> None: | |
| """Remove entries that have passed their forget window.""" | |
| if self._forget_after is not None: | |
| self._visited = [ | |
| (pos, when) for pos, when in self._visited | |
| if not self._expired(when, now) | |
| ] |
| def _decode_base64_image(b64: str) -> Image: | ||
| bgr_array = TurboJPEG().decode(base64.b64decode(b64)) | ||
| return Image(data=bgr_array, format=ImageFormat.BGR) |
There was a problem hiding this comment.
_decode_base64_image has no error handling for invalid input
If initial_image is not valid base64 or is not a JPEG, base64.b64decode or TurboJPEG().decode will raise an uncaught exception that propagates out of the track_subject skill handler. This happens outside the monitor loop's exception guard, so the raw exception surfaces to the caller rather than a graceful error response.
| def _decode_base64_image(b64: str) -> Image: | |
| bgr_array = TurboJPEG().decode(base64.b64decode(b64)) | |
| return Image(data=bgr_array, format=ImageFormat.BGR) | |
| def _decode_base64_image(b64: str) -> Image | None: | |
| try: | |
| bgr_array = TurboJPEG().decode(base64.b64decode(b64)) | |
| return Image(data=bgr_array, format=ImageFormat.BGR) | |
| except Exception: | |
| return None |
| class QwenChinaVlModelConfig(QwenVlModelConfig): | ||
| """Qwen VL config defaulting to a current China-endpoint model.""" | ||
|
|
||
| model_name: str = os.getenv("PAWTRACK_VLM_MODEL", _DEFAULT_MODEL) |
There was a problem hiding this comment.
PAWTRACK_VLM_MODEL is evaluated at class-definition time, not at instantiation
os.getenv(...) runs when the class body is parsed, so the env var is frozen at import time. Any process that sets PAWTRACK_VLM_MODEL after importing pawtrack.qwen_china will not see the change.
| class QwenChinaVlModelConfig(QwenVlModelConfig): | |
| """Qwen VL config defaulting to a current China-endpoint model.""" | |
| model_name: str = os.getenv("PAWTRACK_VLM_MODEL", _DEFAULT_MODEL) | |
| class QwenChinaVlModelConfig(QwenVlModelConfig): | |
| """Qwen VL config defaulting to a current China-endpoint model.""" | |
| model_name: str = _DEFAULT_MODEL | |
| def __init__(self, **data): # type: ignore[override] | |
| data.setdefault("model_name", os.getenv("PAWTRACK_VLM_MODEL", _DEFAULT_MODEL)) | |
| super().__init__(**data) |
2 participants
Problem / Project Description
a DimOS / Unitree Go2 hackathon demo where the robot automously patrols, finds a user-described target such as “
a person sitting on a chair”, approaches safely, waves hello, records the visit, and resumes exploring utilizing dimos' built in agentic ability and custom built detection and react facility .
Solution / Technical Highlights
greeter-agenticblueprint and autonomous demo path.How to Test
To setup environment for running
To run tests in the repo which includes unit test and integration tests:
To run the stack on a real hardware, use :
Start the agenticc loop
dimos mcp call start_greeting --arg target="a person sitting on a chair"Stop the behavior:
Video link
https://drive.google.com/file/d/1JEZV0CbeujOiTJJuaKiz5LULYm7R0eWt/view?usp=drive_link
Contributor License Agreement